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Intrusion and privacy: Exploiting unlabeled data in ensemble methods 
Kristin P. Bennett, Ayhan Demiriz, Richard Maclin 

July 2002 Proceedings of the eighth ACM SIGKDD international conference on 

Knowledge discovery and data mining 
Publisher: ACM Press 

Full text available: 'g| pdf(719.46 KB) Additional Information: full citation , abstract , references , index terms 

An adaptive semi-supervised ensemble method, ASSEMBLE, is proposed that constructs 
classification ensembles based on both labeled and unlabeled data. ASSEMBLE alternates 
between assigning "pseudo-classes" to the unlabeled data using the existing ensemble 
and constructing the next base classifier using both the labeled and pseudolabeled data. 
Mathematically, this intuitive algorithm corresponds to maximizing the classification 
margin in hypothesis space as measured on both the labeled and unlabel ... 



Keywords: boosting, classification, ensemble learning, semi-supervised learning 

2 The principled design of large-scale recursive neural network architectures-dag-rnns Q 
and the protein structure prediction problem 

Pierre Baldi, Gianluca Pollastri 

December 2003 The Journal of Machine Learning Research, volume 4 
Publisher: MIT Press 

Full text available: ^ pdf(231.40 KB) Additional Information: full citation , abstract , references , index terms 

We describe a general methodology for the design of large-scale recursive neural network 
architectures (DAG-RNNs) which comprises three fundamental steps: (1) representation 
of a given domain using suitable directed acyclic graphs (DAGs) to connect visible and 
hidden node variables; (2) parameterization of the relationship between each variable and 
its parent variables by feedforward neural networks; and (3) application of weight-sharing 
within appropriate subsets of DAG connections to capture s ... 

3 Genetic algorithms: Evolvin g neural network ensembles for control problems 
^ David Pardee, Michael Ryoo, Risto Miikkulainen 

June 2005 Proceedings of the 2005 conference on Genetic and evolutionary 

computation GECCO '05 
Publisher: ACM Press 

Full text available: "gj pdf(121.49 KB) Additional Information: full citation , abstract , references , index terms 
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In neuroevolution, a genetic algorithm is used to evolve a neural network to perform a 
particular task. The standard approach is to evolve a population over a number of 
generations, and then select the final generation's champion as the end result. However, 
it is possible that there is valuable information present in the population that is not 
captured by the champion. The standard approach ignores all such information. One 
possible solution to this problem is to combine multiple individuals fro ... 

Keywords: ensembles, genetic algorithms, neural networks, reinforcement learning 



4 A pplication of neural networks to biological data mining: a case study in protein 




sequence classification 

Jason T. L Wang, Qicheng Ma, Dennis Shasha, Cathy H. Wu 

August 2000 Proceedings of the sixth ACM SIGKDD international conference on 

Knowledge discovery and data mining 
Publisher: ACM Press 

Full text available: 'g| pdf(181.04 KB) Additional Information: full citation , references , citings . Index terms 




Keywords: bioinfornnatics, biological data mining, feature extraction from protein data, 
machine learning, neural networks, sequence alignment 



Desi g nin g committees of models through deliberate weightin g of data points ^ 

Stefan W. Christensen, Ian Sinclair, Philippa A. S. Reed 

December 2003 The Journal of Machine Learning Research, volume 4 

Publisher: MIT Press 

Full text available: pdf(205.46 KB) Additional Information: full citation , abstract , references , index terms 

In the adaptive derivation of mathematical models from data, each data point should 
contribute with a weight reflecting the amount of confidence one has in it. When no 
additional information for data confidence is available, all the data points should be 
considered equal, and are also generally given the same weight. In the formation of 
committees of models, however, this is often not the case and the data points may 
exercise unequal, even random, influence over the committee formation. In this ... 

Contributed articles on online , interactive , and anytime data mining: MobiMine: Q 
monitorin g the st ock market from a PDA 

Hlllol Kargupta, Byung-Hoon Park, Sweta PIttie, Lei Liu, Deepali Kushraj, Kakali Sarkar 
January 2002 ACM SIGKDD Explorations Newsletter, Volume 3 issue 2 
Publisher: ACM Press 

Full text available: ^ pclf(1.16 MB) Additional Information: full citation , abstract , references , citings 

This paper describes an experlnnental mobile data mining system that allows intelligent 
monitoring of time-critical financial data from a hand-held PDA. It presents the overall 
system architecture and the philosophy behind the design. It explores one particular 
aspect of the system— automated construction of personalized focus area that calls for 
user's attention. This module works using data mining techniques. The paper describes 
the data mining component of the system that employs a novel Four ... 

Cluster ensembles — a knowled g e reuse frannework for connbinina multiple partitions Q 
Alexander Strehl, Joydeep Ghosh 

March 2003 The Journal of Machine Learning Research, volume 3 
Publisher: MIT Press 

Additional Information: 
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Full text available: ^Ddff842.50 KB) full citation , abstract , references , citings , index 

terms 

This paper introduces the problem of combining multiple partitionings of a set of objects 
into a single consolidated clustering without accessing the features or algorithms that 
determined these partitionings. We first identify several application scenarios for the 
resultant 'knowledge reuse' framework that we call cluster ensembles. The cluster 
ensemble problem is then formalized as a combinatorial optimization problem in terms of 
shared mutual information. In addition to a direct ... 

Keywords: cluster analysis, clustering, consensus functions, ensemble, knowledge reuse, 
multi-learner systems, mutual information, partitioning, unsupervised learning 



8 Learning Ensembles from Bites: A Scalable and Accurate Approach 
Nitesh V. Chawla, Lawrence 0. Hall, Kevin W. Bowyer, W. Philip Kegelmeyer 
December 2004 The Journal of Machine Learning Research, volume 5 
Publisher: MIT Press 

Full text available: ^ pdf{3.34 MB) Additional Information: full citation, abstract , references , index terms 

Bagging and boosting are two popular ensemble methods that typically achieve better 
accuracy than a single classifier. These techniques have limitations on massive data sets, 
because the size of the data set can be a bottleneck. Voting many classifiers built on small 
subsets of data ("pasting small votes") is a promising approach for learning from massive 
data sets, one that can utilize the power of boosting and baggirig. We propose a 
framework for building hundreds or thousands of such classifie ... 

9 Neural networks and dynamic complex systems 

Geoffrey Fox, Wojtek Furmanski, Alex Ho, Jeff Koller, Peter Simic, Isaac Wong 

March 1989 Proceedings of the 22nd annual symposium on Simulation ANSS '89 

Publisher: IEEE Computer Society Press 

Full text available: 'g| pdf(1.44 MB ) Additional Information: full citation , abstract , references , index terms 

We describe the use of neural networks for optimization and inference associated with a 
variety of complex systems. We show how a string formalism can be used for parallel 
computer decomposition, message routing and sequential optimizing compilers. We 
extend these ideas to a general treatment of spatial assessment and distributed artificial 
intelligence. 

10 Information access and retrieval (lAR): Text classification based on data partitioning 
^ and parameter vary ing ensembles 

^ Yan-Shi Dong, Ke-Song Han 

March 2005 Proceedings of the 2005 ACM symposium on Applied computing SAC '05 
Publisher: ACM Press 

Full text available: ^ pdf(231.21 KB) Additional Information: full citation , abstract , references , index terms 

Support vector machines (SVI^) are among the best text classifiers so far. Meantimes, 
ensembles of classifiers are proven to be effective on many domains. It is expected that 
ensembles of SVM classifiers could achieve better performance. In this paper two types of 
. ensembles on SVM classifiers, the data partitioning ensembles and heterogeneous 
ensembles, have been proposed and experimentally evaluated on three well-accepted 
collections. Major conclusions are that disjunct partitioning ensembles wi ... 

Keywords: ensemble, support vector machines, text classification 
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neural network model for software engineerin g ap plications 

K. K. Aggarwal, Yogesh Singh, Pravin Chandra, Manimala Puri 

July 2005 ACM SIGSOFT Software Engineering Notes, volume 30 issue 4 

Publisher: ACM Press 

Full text available: ^ pdf(412.13 KB) Additional Information: full citation , abstract , references , index terms 

Software Engineering as a discipline emerged in response to the software crisis perceived 
by the industry. It is a well known fact that at the beginning of any project, the software 
industry needs to know how much will it cost to develop and what would be the time 
required. Resource estimation in software engineering is more challenging than resource 
estimation in any other industry. A number of resource estimation methods are currently 
available and the neural network model is one of them. This ... 

Keywords: neural network, resource estimation, software engineering, training algorithm 



12 Research track posters: Model compression 

Cristian BuciluD, Rich Caruana, Alexandru Niculescu-Mizil 

August 2006 Proceedings of the 12th ACM SIGKDD international conference on 

Knowledge discovery and data mining KDD '06 
Publisher: ACM Press 

Full text available: "gl pdffl.OQ MB) Additional Information: full citation , abstract , references , index terms 

Often the best performing supervised learning models are ensembles of hundreds or 
thousands of base-level classifiers. Unfortunately, the space required to store this many 
classifiers, and the time required to execute them at run-time, prohibits their use in 
applications where test sets are large (e.g. Google), where storage space is at a premium 
(e.g. PDAs), and where computational power is limited (e.g. hea-ring aids). We present a 
method for "compressing" large, complex ensembles into smaller ... 



Keywords: model compression, supervised learning 



''2 Round robin classification 
Johannes Furnkranz 

March 2002 The Journal of Machine Learning Research, volume 2 
Publisher: MIT Press 

Full text available* "SI pdf (250.25 KB) A^^'^'^nal Information: full citation , abstract , references , citings, index 

terms 

In this paper, we discuss round robin classification (aka pairwise classification), a 
technique for handling multi-class problems with binary classifiers by learning one 
classifier for each pair of classes. We present an empirical evaluation of the method, 
implemented as a wrapper around the Ripper rule learning algorithm, on 20 multi-class 
datasets from the UCI database repository. Our results show that the technique is very 
likely to improve Ripper's classification accuracy without having a hi ... 

Keywords: class binarization, ensemble techniques, inductive rule learning, multi-class 
problems, pairwise classification 



14 Technical Correspondence: A neural net compiler system for hierarchical 
^ org anization 

Rajeev Kumar 

February 2001 ACM SIGPLAN Notices, volume 36 issue 2 
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Publisher: ACM Press 

Full text available: 'Q pdf(954.76 KB) Additional Information: full citation , abstract , references 

We present a language framework for handling arbitrarily complex neural computations. 
The software architecture - which we call an >*rtificial Weural Network Compiler for 
Hierarchical Organization (>A/VCA/0/^) - facilitates network hierarchy and simpler sub- . 
mappings. We define a Net Definition Language (IMDL) which is implemented in object- 
oriented programming paradigm; a trained network is decompiled bac ... 

Keywords: compiler-decompiler, hierarchical networks, neural net definitions, neural 
programming language, superneuron 



'1 5 Research track: Minin g concept-drifting data streams using ensemble classifiers 
Halxun Wang, Wei Fan, Philip S. Yu, Jiawei Han 

August 2003 Proceedings of the ninth ACM SIGKDD international conference on 

Knowledge discovery and data mining 
Publisher: ACM Press 

Full text available* IS pdf(234 1 3 KB) A^^'*'^'^^' Information: full citation , abstract , references , citings , index 

1^. terms 

Recently, mining data streams with concept drifts for actionable insights has become an 
important and challenging task for a wide range of applications including credit card fraud 
protection, target marketing, network intrusion detection, etc. Conventional knowledge 
discovery tools are facing two challenges, the overwhelming volume of the streaming 
data, and the concept drifts. In this paper, we propose a general framework for mining 
concept-drifting data streams using weighted ensemble classifi ... 

Keywords: classifier, classifier ensemble, concept drift, data streams 



^6 S pecial issue on learnin g fronn innbalanced datasets: Minority report in fraud 
^ detection: classification of skewed data 

Clifton Phua, Damminda Alahakoon, Vincent Lee 

June 2004 ACM SIGKDD Explorations Newsletter, volume 6 issue i 

Publisher: ACM Press 

Full text available: ^ pdf (262.38 KB) Additional Information: full citation , abstract , references , citings 

This paper proposes an innovative fraud detection method, built upon existing fraud 
detection research and Minority Report, to deal with the data mining problem of skewed 
data distributions. This method uses backpropagatlon (BP), together with naive Bayesian 
(NB) and C4.5 algorithms, on data partitions derived from minority oversampling with 
replacement. Its originality lies In the use of a single meta-classifier (stacking) to choose 
the best base classifiers, and then combine these base ... 

Keywords: fraud detection, meta-learning, multiple classifier systems 



''^ Building predictors from vertically distributed data Q 
Sabine McConnell, David B. Skillicorn 

October 2004 Proceedings of the 2004 conference of the Centre for Advanced Studies 

on Collaborative research 
Publisher: IBM Press 

Full text available: g pdf(1 84.78 KB) Additional Information: full citation , abstract , references , index terms 

Due in part to the large volume of data available today, but more importantly to privacy 
concerns, data are often distributed across institutional, geographical and organizational 
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boundaries rather than being stored in a centralized location. Data can be distributed by 
separating objects or attributes: in the homogeneous case, sites contain subsets of 
objects with all attributes, while in the heterogeneous case sites contain subsets of 
attributes for all objects. Ensemble approaches combine t ... 

18 Hy percube al g orithms for neural network simulation: the Crystal Accumulator and 
the Cr/stal Router 
G. C. Fox, W. Furmanski 

January 1988 Proceedings of the third conference on Hypercube concurrent 

computers and applications: Architecture, software, computer systems, 
and general issues - Volume 1 

Publisher: ACIVl Press 

Full text available* •S pdf (466.Q0 KB) Additional Information: full citation , abstract , references , citings , index 

terms 

We discuss communication algorithms relevant for neural network modeling on distributed 
memory concurrent computers with a hypercube topology. Full, intermediate (medium 
range) and sparse network connectivities are analyzed. We point out that the flexible 
hypercube topology allows for the efficient implementation of the broad class of network 
algorithms with variety of connectivity patterns. We find algorithms index, crystaLrouter, 
fold an ... 



The distributed boosting algorithm 
Aleksandar Lazarevic, Zoran Obradovic 

August 2001 Proceedings of the seventh ACM SIGKDD international conference on 
Knowledge discovery and data mining 

Publisher: ACM Press 

Full text available: pdf(625.23 KB) Additional Information: full citation , abstract , references , index terms 

In this paper, we propose a general framework for distributed boosting intended for 
efficient integrating specialized classifiers learned over very large and distributed 
homogeneous databases that cannot be merged at a single location. Our distributed 
boosting algorithm can also be used as a parallel classification technique, where a 
massive database that cannot fit into main computer memory is partitioned into disjoint 
subsets for a more efficient analysis. In the proposed method, at each boost ... 



Keywords: Boosting, classifier ensembles, distributed learning 



20 Attribute Clustering for Grouping. Selection, and Classification of Gene Expression Q 
Data 

Wai-Ho Au, Keith C. C. Chan, Andrew K, C. Wong, Yang Wang 

April 2005 IEEE/ACM Transactions on Computational Biology and Bioinformatics 

(TCBB), Volume 2 Issue 2 
Publisher: IEEE Computer Society Press 

Full text available: ^ pdf(2.58 MB) Additional Information: full citation , abstract , references , index terms 

This paper presents an attribute clustering method which is able to group genes based on 
their interdependence so as to mine meaningful patterns from the gene expression data. 
It can be used for gene grouping, selection, and classification. The partitioning of a 
relational table into attribute subgroups allows a small number of attributes within or 
across the groups to be selected for analysis. By clustering attributes, the search 
dimension of a data mining algorithm is reduced. The reduction of ... 

Keywords: Data mining, attribute clustering, gene selection, gene expression 
classification, microarray analysis. 
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Genetic Al g orithm based Selective Neural Network Ensemble - Zhou ,.. 
Neural network ensemble is a learning paradigm where several neural networks are jointly 
used to solve a problem. In this paper, the relationship between ... 
citeseer.ist.psu.edu/zhou01genetic.html - 27k - Cached - Similar p ages 

A Genetic Algorithm Approach for Creating Neural-Network Ensembles 
A neural. network ensemble is a successful technique where the outputs of a set of ... A 
Genetic Algorithm Approach for Creating Neural-Network Ensembles ... 
citeseer.ist.psu.edu/opitz99genetic,html - 22k - Cached - Similar pages 
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uses a neural network ensemble to successfully identify a topic, instead of ... 
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Artificial neural network - Wikipedia, the free encyclopedia 
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